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Abstract. In various astronomical projects it is crucial to have coordinates 
indexed tables. All sky optical and IR catalogues have up to 1 billion objects 
that will increase with forthcoming projects. Also partial sky surveys at various 
wavelengths can collect information (not just source lists) which can be saved in 
coordinate ordered tables. Selecting a sub-set of these entries or cross- matching 
them could be un-feasible if no indexing is performed. Sky tessellation with 
various mapping functions have been proposed. It is a matter of fact that the 
astronomical community is accepting the HTM and HEALPix schema as the de- 
fault for object catalogues and for maps visualization and analysis, respectively. 
Within the MCS library project, we have now made available as MySQL-callable 
functions various HTM and HEALPix facilities. This is made possible thanks 
to the capability offered by MySQL 5.1 to add external plug-ins. The DIE (Dy- 
namic Indexing Eacilities) package distributed within the MCS library, creates 
and manages a combination of Views, Triggers, DB-engine and plug-ins allowing 
the user to deal with database tables indexed using one or both these pixelisation 
schema in a completely transparent way. 



1. Introduction 

Spatial indexes has always been an important issue for multi dimensional data 
sets in relational databases (DBs), in particular for those dealing with spherical 
coordinates, e.g. latitude/longitude for Earth locations or RA/Dec for celestial 
objects. Some DB servers offer built-in capabilities to create indexes on these 
(coordinate) columns which consequently speed up the execution of queries in- 
volving them. However 1. the use of these facilities could be not easy, 2. they 
typically use a syntax quite different from the astronomical one, 3. their perfor- 
mance is inadequate for the astronomical use. 

Within the MCS library project (Calderone & Nicastro 2007; Nicastro & 
Calderone 2006, 2007; ross.iasfbo.inaf.it/MCS/) we have implemented the 
DIF package, a tool which performs and manages in a fully automatic way the 
sky pixelisation with both the HTM (Kunszt et al. 2001) and HEALPix (Gorski 
et al. 2005) schema. Using a simple tool, any DB table with sky coordinates 
columns can be easily indexed. This is achieved by using the facilities offered by 
the MySQL DB server (which is the only server MCS supports at the moment), 
i.e. triggers, views and plugins. Having a table with sky coordinates, the user 
can make it fully indexed in order to perform quick queries on rectangular and 
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circular regions (cone) or to create an HEALPix map file. An SQL query to 
select objects in a cone will look like this: SELECT * FROM MyCatalogue WHERE 
EntriesInCone(20, 30, 5), where (20,30) are the coordinates of the center 
in degrees and 5 is the radius in arcmin. The important thing to note is that 
the DB manager needs to supply only a few parameters in the configuration 
phase, whereas the generic user does not need to know anything about the sky 
pixelisation either for SELECT or INSERT or UPDATE qTicrics. It also demonstrates 
that there is no need to extend standard SQL for astronomical queries (see 
ADQL), at least if MySQL is used as DB server. 



2. Indexing on a sphere 

In terms of DB table indexing, mapping a sphere with a pixel scheme means 
transforming a 2-d into a 1-d space, consequently a standard B-tree index 
can be created on the column with the pixel IDs. On a large astronomical 
table, depending on the "depth" of the pixelisation, this could lead to a gain 
of a 4-5 orders of magnitude in search efficiency. The HTM and HEALPix 
schema arc widely used in Astronomy and arc now well mature to be considered 
as candidates for indexing tables containing astronomical data. They are both 
open source and distributed as C++ libraries. HTM uses triangular pixels which 
can recursively be subdivided into four pixels. The base pixels arc 8, 4 for each 
hemisphere. These "trixels" are not equal-area but the indexing algorithm is 
very efficient for selecting point sources in catalogues. HEALPix uses equal-area 
pseudo-square pixels, particularly suitable for the analysis of large-scale spatial 
structures. The base pixels are 12. Using a 64 bit long integer to store the index 
IDs leads to a limit for the pixels size of about 7.7 and 0.44 milli-arcsec on a side 
for HTM and HEALPix, respectively. Being able to quickly retrieve the list of 
objects in a given sky region is crucial in several projects. For example himting 
for transient sources like GRBs requires fast catalogues lookup so to quickly 
cross match known sources with the detected objects. The IR/optical robotic 
telescope REM (Nicastro & Calderone 2006) uses HTM indexed catalogues to 
get the list of objects in 10' x 10' regions. In this case accessing one billion 
objects catalogues like the GSC2.3 takes some 10 msec. Having a fully automatic 
HTM and HEALPix indexing would be crucial for the management of the DBs 
of future large missions like Gaia. Also the Virtual Observatory project would 
greatly benefit from adopting a common indexing scheme for all the various types 
of archive it can manage. The relevant parameters for the two pixelisations are: 

HTM HEALPix 

nI^: 8x4'^ 12 X iVj^^ (where iV^ide = 2^=) 

ID range: [iVpix , 2 x N^^^ - 1] [0 , A^pix - 1] 

Max iVpix: ~ 9.0 x 10^^ ~ 3.5 x 10^^ 

Max res. ("): 7.7 x lO'^ 3.9 x 10"^ (fJpix = 7r/(3 x N^.^^)) 



(depth): [0 ,25]; k (order <;=^ resolution parameter): [0 ,29] 



As mentioned the maximum resolution is related to the usage of 64 bit integers 
and it is intrinsic to the HTM and HEALPix C++ libraries. 
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3. The MCS DIF package 

MCS is a set of C++ high level classes aimed at implementing an application 
server, that is an application providing a service over the network. MCS provides 
classes to interact with, manage and extend a MySQL DB server. The included 
MyRO package allows a per row management of DB grants whereas the DIF 
package allows the automatic management of sky pixelisation with the HTM 
and HEALPix schema. See the lMCS web sitel for more information. 

To enable DIF, when installing MCS it is enough to give to the configure 
script the two options — enable-dif — with-niysql-source=PATH where PATH 
is the path to the MySQL source directory. The HTM and HEALPix C++ 
libraries are included in the DIF package. A DB named DIF will be created 
containing an auxiliary table tbl and a virtual table dif which is dynamically 
managed by the DIF DB engine. Now let's assume one has a DB MyDB with a 
table MyCat containing the two coordinates column RAcs and DECcs representing 
the centi-arcsec converted J2000 equatorial coordinates (this requires 4 bytes 
instead of the 8 necessary for a double value). To make the table manageable 
using both the HTM and HEALPix pixelisation schema it is enough to give the 
command: 

dif —index-both MyDB MyCat 6 8 "RAcs/3.6E5" "DECcs/3 . 6E5" 

where dif is the name of the script used to perform administrative tasks related 
to DIF-handled tables, 6 is the HTM depth and 8 is the HEALPix order whereas 
the (1) selects the RING (NESTED) scheme. The last two parameters are the 
SQL expressions which convert to degrees the coordinate values contained in the 
table fields RAcs and DECcs. If the coordinates where already degrees, then it 
would have been enough to give their names, e.g. dif . . . RA DEC. The MySQL 
root password is needed. In a future release we'll add the possibility to perform 
simple cross matching between (DIF managed) catalogues. Having an HTM 
indexed catalogue, the query string to obtain the list of objects in a circular 
region centred on a = 60° and 5 = 30° with radius 40' will be: 

SELECT * FROM MyCat_htin WHERE DIF_HTMCircle (60 , 30 , 40) ; 
note the table name _htni suffix which is needed to actually access the view 
handled by DIF. For a rectangle with the same centre and sides 50' along the a 
axis and 20' along the 5 axis: 

SELECT * FROM MyCat_htm WHERE DIF_HTMRect (60 , 30 , 50 , 20) ; 
giving only three parameters would imply a square selection. Having chosen to 
use both HTM and HEALPix indexing, one could request all the HEALPix IDs 
of the objects in a 50' square by using an HTM function: 

SELECT healpID FROM MyCat_htm WHERE DIF_HTMRect (60 , 30 , 50) ; 
To simply get the IDs of the pixels falling into a circular /rectangular region 
one can simply SELECT id FROM DIF. dif WHERE . . ., i.e. no particular DIF 
managed table is required. To obtain the order 10 IDs in RING scheme one can 
calculate them on the fly: 

SELECT DIF_HEALPLookup (0,10, RAcs/3 . 6E5 , DECcs/3 . 6E5) 
FROM MyCat_htm WHERE DIF_HTMCircle (60 , 30 , 20) ; 
Giving 1 instead of would give NESTED scheme IDs. Having RA and DEC in 
degrees one would simply type (0, 10,RA,DEC) . If one has just the HEALPix 
IDs then entries on a circular region can be selected like in: 
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MAIN Table 

Table containing data, coordinates and relative htnilD 





Ra 


Dec 




htmID 



DIF.dif Table 

A reference table whose contents are dynamically 
generated on the base of tiser search criteria 



The join operation is very fast because / JOIN 
it is based on the indexed field htmlD \ htmID = ID 




MAIN_htm VIEW 

Front-end view to access the underlying indexing facility 
This view has the same structure as the table MAIN 





Ra 


Dec 




htmID 



Search criteria generates 
reference IDs in table DIF.dif 



Example query: SELECT * FROM MAIN_htm WHERE (DIFJTMCircle ( 10 , 20, 3) ; 

Z 





Center coordinates 




Figure 1 . This diagram summarises the way DIF works for the HTM index- 
ing case. The extra column htmID is added to the original table MAIN and 
a view is created which performs a join with the dynamical table DIF.dif. 
Triggers to automatically manage INSERT and UPDATE c^ueries are also cre- 
ated. We recall that MCS allows users to interact with such MySQL server 
using programs written in any language. 



SELECT * FROM MyCat_healp WHERE DIF_HEALPCircle (60 , 30 , 40) ; 
note the table name _healp suffix. Rectangular selections for only-HEALPix 
indexed tables will be available in the future. The current list of functions is: 
DIF_HTMCircle, DIF_HTMRect, DIF_HTMRectV, DIF_HEALPCircle, 
DIF_HTMLookup, DIF_HEALPLookup, DIF_Sphedist. 

DIF_HTMRectV accepts the four corners of a rectangle which can then have any 
orientation in the sky. DIF_Sphedist calculates the angular distance of two 
points on the sphere by using the haversines formula. A first version of IDL user 
contributed library and demo programs aimed at producing HEALPix maps 
from the output of SQL queries is available at the lMCS web sitel 
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